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Abstract — The multi-terminal rate-distortion problem has 
been studied extensively. Notably, among these, Tung and House- 
wright have provided the best known inner and outer bounds for 
the rate region under certain distortion constraints. In this paper, 
we first propose an outer bound for the rate region, and show that 
it is tighter than the outer bound of Tung and Housewright. Our 
outer bound involves some n-letter Markov chain constraints, 
which cause computational difficulties. We utilize a necessary 
condition for the Markov chain constraints to obtain another 
outer bound, which is represented in terms of some single- 
letter mutual information expressions evaluated over probability 
distributions that satisfy some single-letter conditions. 

I. The Multi-Terminal Rate-Distortion Problem 

Ever since the milestone paper of Wyner and Ziv [1] 
on the rate-distortion function of a single source with side 
information at the decoder, there has been a significant amount 
of efforts directed towards solving a generalization of this 
problem, the so called multi-terminal rate-distortion problem. 
Despite these efforts, the problem remains open to this day. 
Among all the attempts on this difficult problem, the works by 
Tung [2] and Housewright [3] (see also [4]) provide the best 
inner and outer bounds so far for the rate-distortion region. 

The multi-terminal rate-distortion problem can be formu- 
lated as follows. Consider a pair of discrete memoryless 
sources (U, V), with joint distribution p(u, v) defined on the 
finite alphabet U x V. The reconstruction of the sources 
are built on another finite alphabet IA x V. The distortion 
measures are defined as d\ : U x U i — ► K + U {0} and 
d 2 : V x V i — > M + U {0}. Assume that two distributed 
encoders are functions fx : U n i — > {1,2, . ..,Mi} and 
f 2 : V™ i — ► {1,2,..., M 2 } and a joint decoder is the function 
g : {1, 2, ... , Mi} x {1, 2, . . . , M 2 } i — > U n x V", where n is 
a positive integer. A pair of distortion levels D = (Dx, D 2 ) is 
said to be R-attainable, for some rate pair R = (Rx,R 2 ), if 
for all e > and S > 0, there exist, some positive integer n and 
a set of distributed encoders and joint decoder (fx, f 2 , g) with 
rates log 2 Mi, \ log 2 M 2 ) = (Rx + 5, R 2 + 5), such that 
the distortion between the sources ({/", V n ) and the decoder 
output (U n ,V n ) satisfies (Edx(U n , V n ), Ed 2 (V n , V n )) < 
(Di + e,D 2 + e) 1 where d 1 (U n ,U n ) 4 i^^Di,^) 
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'By (A, B) < (C, D), we mean both A < B and C < D, and (A, B) < 
(C, D) is defined in the similar manner. 



and d 2 (V n ,V n ) 4 I d 2 {Vu The problem here is 
to determine, for a fixed D, the set 7£(D) of all rate pairs R, 
for which D is R-attainable. 

We restate the outer bound provided in [2] and [3] in the 
following theorem. 

Theorem 1 ([2], [3]) K(D) C ft out ,i(D), where ft ouM (D) 
is the set of all R such that there exists a pair of discrete 
random variables {X\,X%), for which the following three 
conditions are satisfied: 

1) The joint distribution satisfies 



U 

u 



V 
V 



2) The rate pair satisfies 

Ri > I(U, V 
Ri > I(U, V 
Ri + R 2 > 1(17, V 



-+X 2 

Xx\X 2 ) 
X 2 \X X ) 
Xx,X 2 ) 



3) There exists (U(X U X 2 ), V(X U X 2 )) 
(Ed x {U,U),Ed 2 {V,V)) < D. 



(1) 
(2) 

(3) 
(4) 
(5) 

such that 



An inner bound is also given in [2] and [3] as follows. 

Theorem 2 ([2], [3]) 11(D) D K in (T>), where TZ in {B) is the 
set of all R such that there exists a pair of discrete random 
variables (X\,X 2 ), for which the following three conditions 
are satisfied: 

1) The joint distribution satisfies 



Xx -» U -» V -» X 2 

2) The rate pair satisfies 

Rx >I(U,V;Xx\X 2 ) 
R2 > I{U,V;X 2 \Xx) 
Rx + Ri >I(U,V;Xx,X 2 ) 



(6) 

(7) 
(8) 
(9) 



3) There exists (U(Xx,X 2 ),V(Xx,X 2 )) such that 
(Edx(U,U),Ed 2 (V,V)) <D. 

We note that the inner and outer bounds agree on both 
the second condition, i.e., the rate constraints in terms of 



some mutual information expressions, and the third condition, 
i.e., the reconstruction functions. However, the first condition 
in these two bounds constraining the underlying probability 
distributions p(x\,x 2 \u,v) are different. It is easy to see that 
the Markov chain condition in the inner bound, i.e., X\ — > 
U — > V — > X 2 , implies the Markov chain conditions in the 
outer bound, i.e., X\ — > U — > V and U — > V — > X 2 . Hence, 
if we define 

<5 ou t,i = {p(ori,a;2|M,w) : Xi -> [7 -> V and [7 -> V -> X 2 } 

(10) 

5 in = Mxi,x 2 | U , V ) : X 1 - (7 -> V -> X 2 } (11) 
then, 

<5 ou t,i 2 <5j„ (12) 

Using the time-sharing argument, a convexification of the 
inner bound lZi n (D) yields another inner bound 1Z' in (D), 
which is larger. This new inner bound may be expressed as a 
function of Sj n and D as follows, 

K(T>) 2 n' m (V) = T(S m , D) D K in (V) (13) 

where, using a time sharing random variable Q, which is 
known by encoders and decoder, ,F(5» n ,D) is defined as, 

^(5j„,D)4 (J C(p) (14) 
peP(S« n ,D) 

p =p(a;i,.T 2 ,g|M,w) = p 9 (a;i, z 2 |u, u)p(g') (15) 

!p 9 (a;i,a; 2 |u,w) e S in ; } 

p: i^Xi.Xa.QJ.^Xi.Xa.Q)), \ 

s.t. (^di(f/,f/),i;d 2 (v,y)) <d J 

(16) 

f i2i >/(^,y;Xi|X 2 ,Q) ] 

C{p)±{ (J?i,i? 2 ) : i? 2 >/(^,y;X 2 |Xi,Q) } 

(17) 

In [3], it was shown that H ut,i(D) is convex, which means 
that 1Z out ^(D) can also be represented as a function of «S out; i 
and D as follows. 

7e out ,i(D) =JF(5 out ,i,D) (18) 

Therefore, we conclude that the gap between the inner and 
the outer bounds comes only from the difference between the 
feasible sets of the probability distributions p(xi,x 2 \u,v). In 
the next section, we will provide a tighter outer bound for the 
rate region in the sense that it can be represented using the 
same mutual information expressions, however, on a smaller 
feasible set for p(x\, x 2 \u, v) than lZ out s(D). 

II. A New Outer Bound 
We propose a new outer bound as follows. 

Theorem 3 TZ(D) C 72. out , 2 (D), where 7£ out , 2 (D) is the set 
of all R such that there exist some positive integer n, and 
discrete random variables Q,X\ 1 X 2 for which the following 
three conditions are satisfied: 



1) The joint distribution satisfies 

p(u n ,v n ,xi,x 2 ,q) 

n 

=p(q)p(x 1 \u n , q)p{x 2 \v n , q) ~\\p{u %1 v % ) (19) 

»=i 

2) The rate pair satisfies 

fli>/(E/i,Vi;Xi|X 2 ,Q) (20) 
R 2 >I(U 1 ,V 1 ;X 2 \X 1 ,Q) (21) 
Ri+R 2 > I{U 1 ,V 1 ;X 1 ,X 2 \Q) (22) 

where (Ui,Vi) is the first sample of the n-sequences 
(U n ,V n ). 

3) There exists (U(Xi,X 2 ,Q),V(Xi,X 2 ,Q)) such that 
(Ed^U^EthtyV)) < D. 

or equivalently, 

7e out , 2 (D) =J 7 (5 out ,2,D) (23) 

where 

S ut,2 = {pixux^uuv!) : X 1 - U n -^V n ^ X 2 } (24) 

Proof: Consider an arbitrary set of distributed encoders 
and joint decoder (/i, / 2 , g) with reconstructions (U n ,V n ) = 
g(W,Z), where W = fi(U n ) and Z = f 2 (V n ), such that 
(#di(^T>),£d 2 (U™,f")) < (Di + e,L> 2 + e). Here, 
we use i?! = ^Iog 2 (Mi) = ^log 2 (|W|) and i? 2 = 
±log 2 (M 2 ) = ±log 2 (|Z|). 

We define the auxiliary random variables Xu = (W, U % 1 ) 
and X 2i = [Z.V 1 - 1 ). Then, we have 

Iog 2 (Mi) > H(W) 
= I(U n ,V n ;W) 
i) 

> I(U n ,V n ;W\Z) 

n 

= J2 I (U i ,V i ;W\Z, IP- 1 ,^- 1 ) 

i=l 
n 

= Y,I{U i ,V i ;W,Z\U i -\V i - 1 ) 
i=i 

-/(tfi.VfcZltf*- 1 ,^*- 1 ) 

n 

^ ^ 7(t7i, V t ; W, Zp 1 - 1 ,^- 1 ) - I(Ui, V i; Zp^ 1 ) 

i=l 
n 

i=l 

n 

% HUi, Vi-, W, Z, W- 1 ^- 1 ) - I{Ui, v t] z\v- v ) 

i=l 
n 

= Y / I(U l ,V l ;W,W- 1 \Z, V'- 1 ) 

i=l 
n 

= ^7(^,I/ i ;X li |X 2i ) (25) 

i=l 



where 

1) follows from the fact that W -»• U n -» V"" -► Z. We 
observe that the equality holds when W is independent 
of Z; 

2) from the fact that 

p(z\ui, Vi, v 1 ' 1 ) = p(z\ui, v i ,u' l ~~ L ,v % ~~ L ) (26) 

3) from the memoryless property of the sources. 
Using a symmetrical argument, we obtain 

n 

log 2 (M 2 )>^2l(U i ,V i ;X 2i \X li ) (27) 

Moreover, 

log 2 (MiM 2 ) > H(W,Z) 

=I(U n ,V n ;W,Z) 

n 

=J2 H (Ui,Vi)-H(Ui,Vi\w, z, trSv*- 1 ) 

2=1 

n 

= ^/(C/ i ,Fi;X li ,X 2i ) (28) 
i=i 

We introduce a time-sharing random variable Q, which is 
uniformly distributed on {1, . . . , n] and independent of U n 
and V n . Let the random variables X\ and X2 be such that 

p(xu,X2i\ui,Vi,u~\v~~ l ) = p(x 1 ,X2\ui,vi,u~ 1 ,v~ 1 ,Q = i) 

(29) 

where [M == {[7 X , . . . , CT 4 _i, C/ i+ i, . . . , U n } and V~* is 
defined similarly. Then, 

n 

Y,I{U i ,V i ;X li \X 2i ) = nI(U 1 ,V 1 ;X 1 \X 2 ,Q) (30) 
»=i 

n 

J2l(Ui,V i ;X 2i \X li ) = nI(U 1 ,V 1 ;X 2 \X 1 ,Q) (31) 
i=i 

J2HU t ,V l ;X lt ,X 2i ) = nI(U 1 ,V 1 -X 1 ,X 2 \Q) (32) 

i=l 

The reconstruction pair (U, V) is defined as follows. When 

Q = i, (U, V) = (Ui, Vi), i.e., the i-th letter of (U n , V n ) = 
g(W, Z). (Ui, Vi) is a function of (W, Z), and, therefore, it is 
a function of (X\, X 2 ,Q). Hence, we have that (U,V) is a 
function of (X 1} X 3 , Q), i.e., (l/{X 1 ,X 2 ,Q), V(X 1 ,X 2 ,Q)). 
It is easy to see that 

(EdtiU^Ed^VV)) = (Ed x {U n ,V n ),Ed 2 {V n ,V n )) 
<{D l +e,D 2 + e) (33) 

which completes the proof. ■ 
Next, we state and prove that our outer bound given in 
Theorem|3]is tighter than the outer bound of [2] and [3] given 
in Theorem [2 

Theorem 4 

ftout,i(D) D ft ut, 2 (D) (34) 



Proof: Here, we provide two proofs. First, we prove 
this theorem by construction. For every (Ri,R 2 ) point in 
72-out. 2(0)1 there exist random variables Q,Xi,X 2 satisfying 
dT9l. (Ri,R 2 ) pair satisfying ^3, (EB and and a 

reconstruction pair (U(X 1 ,X 2 ,Q),V(X 1 ,X 2 ,Q)) such that 
(Ed 1 (U,U),Ed 2 (V,V)) < D. According to [3], let X[ = 
(X U Q) and X' 2 = (X 2 , Q). Then, X[ and X' 2 satisfy the first 
condition of Theorem ^ Moreover, 

Rx >I(U,V;X 1 \X 2> Q)=I{U,V;X' l \X' 2 ) (35) 
and similarly, 

R 2 > I(U, V;X 2 \X U Q) = I(U, V; X' 2 \X[) (36) 

and finally, 

R 1 +R 2 >I{U,V;X 1 ,X 2 \Q) 

= H{U,V\Q) - H(U,V\X 1 ,X 2 ,Q) 

= H(U,V) -H(U, V\X!,X 2 ,Q) 

= H{U,V)-H{U,V\X[,X' 2 ) 

= I(U,V;X[,X' 2 ) (37) 

where 1) follows from the fact that Q is independent of (U,V). 
(U,V) is a function of (Xi, X 2 ,Q), and, therefore, it is a 
function of {X[,X' 2 ) = ((X 1 ,Q),(X 2 ,Q)). 

Hence, for every rate pair (R 1: R 2 ) £ 7^. out 2(D), there 
exist random variables X[ , X 2 satisfying the first condition of 
Theorem[2 and (Ri, R 2 ) pair satisfies the second condition of 
Theorem ^ with the distortion satisfying the third condition 
of Theorem^ In other words, (R\, R 2 ) G 7?.out,i(D), proving 
the theorem. 

An alternative proof comes from the comparison of S ou t,i 
and S ou t. 2 , the feasible sets of probability distributions 

p{x x ,x 2 \ui,vxf. We note that X x -> U n -» V n -» X 2 
implies Xx — *■ Ux — » V\ and Ux — > Vx — > X 2 , which means 
that 

S ut,i _> S ou t,2 (38) 

and therefore 

ftout.i(D) = J 7 (5 otlt ,i,D) 2 ^(5 out . 2 ,D) = 7e ut, 3 (D) 

(39) 
■ 

III. A Single-letter Outer Bound 
The proposed outer bound 7£ ut,2(D) is an n-letter bound 
because the feasible set of the probability distribution, S ou t, 2 , 
is characterized by an n-letter Markov chain constraint Xx — > 
U n — > V n — > X 2 , which is practically incomputable when 
n is sufficiently large. In the rest of this section, we will find 
a single-letter necessary condition for this n-letter Markov 
chain. By doing this, we will obtain a single-letter outer bound 
for K(T>). 

We introduce our matrix notation for probability distribu- 
tions [5], [6]. For a pair of discrete random variables X 

'in S ou t,i, the probability distribution is p(xi, X2 \u, v). Here, we just 
rename U = U\ and V = Vi . 



and Y, which take values in X = {xi, x 2 , ■ ■ ■ ,Xk} and 
y = {yij 1/2, ■ ■ ■ , Vi}, respectively, the joint distribution matrix 
Pxy is defined as Pxv{i,j) = Pr{X = x i: Y = y 3 ), 
where Pxy(hj) denotes the (i,j)-th element of the matrix 
Pxy- Similarly, we define Pxy\z as PxY\z(hj) — Pr(X = 
Xi,Y = Uj\Z = z). The marginal distribution of a random 
variable X is defined as a diagonal matrix with Px(i,i) = 
Pr(X — Xi). The vector-form marginal distribution is defined 
as p x (i) = Pr(X — xi), i.e., px = Pxe, where e is a vector 
of all ones, px can also be defined as px = Pxy f° r some 
Y where the size of the alphabet of Y, \y\, is equal to one. A 
column vector px\ z is defined as px\*{i) — Pr(X = x*\Z = 
z), or equivalently, px\z{i) = P X Y\z f° r some Y where the 
size of the alphabet of Y, \y\, is equal to one. We define a 
new quantity, Pxy, as 



Pxy = P x ~ 2 PxyP^ 



(40) 



Since px = Pxy for some Y where the size of the alphabet 
of Y, \y\, is equal to one, we define 



Px 



X 



p 



X 



'Px 



(41) 



The conditional distributions P X y\z an d Px\z can t> e defined 
similarly. 

In [5], we provided a new data processing inequality, a 
necessary condition for a Markov chain, as follows. 

Theorem 5 ([5]) IfX^Y^Z, then 

Ai(Pxz) < MPxy)MPyz) < HPxy) (42) 

where i = 2, . . . , rank(Pxz). an d where Aj(-) denotes the 
i-th largest singular value of a matrix. 

We have also shown in [5] the following lemma. 

Lemma 1 ([5]) For a pair of i.i.d. sequences (X k ,Y k ) char- 
acterized by a joint distribution Pxy, the ordered singular 
values of PxkY k are 

{1,A 2 (/W),...,A 2 (/W),...} 

where the second through the k + 1st singular values are all 
equal to A 2 (Pxf)- 

Combining Theorem|5]and Lemma^ we have the following 
theorem. 

Theorem 6 ([6]) Let (U n ,V n ) be a pair of i.i.d. sequences 
of length n, and let the random variables X\ and X2 
satisfy X x — > U n — > V n — > X 2 . Then, for i = 
2,...,min(|^i|,|^ 2 |), 



Ai(PxxX 2 ) < M{Puv) 

HPx lX2 \ ul ) < MPuv) 

\{P Xl X 2 \vJ<MPuv) 

\{Px iX 2 |ui«i ) < X 2 (Pu V ) 



(43) 
(44) 
(45) 
(46) 



or equivalently 



where 



out,3 



(47) 



(O, (EU, an d (|i6|) are satisfied} (48) 

Proof: The proof of d43t follows from our result in 
[5]. In §4), P Xl x 2 j L u 1 = Px 1 u-l\u 1 Pu-w^\u 1 Pv^x 2 - It 
can be shown that Pjj-' L v n \ui = P'u- 1 v- 1 ® Pv^m w h ere 
® represents the Kronecker product. Since Py 1 ^ ui is a vec- 
tor with singular value 1, Xi(Pu- 1 v r <-\ui) = Aj(P{/-i y-\ ). 
Applying Theorem |5] and |6] J44I is proven. In a simi- 
lar manner, J45> can be shown. In (I46> . Px 1 x 2 \u 1 v 1 = 
Px 1 u-^\u 1 v 1 Pu-W-^\u 1 v 1 Pv-^x 2 \u 1 v 1 - Since (U n ,V n ) is a 
pair of i.i.d. sequences, P(j- 1 v- 1 \u 1 v 1 — Pjj- 1 v- 1 - Then (I46> 
is a direct result of Theorem [5] and |S] ■ 
From the above discussion, we obtain the main result of our 
paper, which is stated in the following theorem. 

Theorem 7 K(T>) C 7e out!3 (D), where 7e out , 3 (D) is the 
set of all R such that there exists some positive integer n, 
and there exist discrete random variable Q independent of 
(U n ,V n ), and discrete random variables Xi,X 2 for which 
the following three conditions are satisfied: 

1) The joint distribution satisfies, for i = 
2,...,rxiin(|^i| I |^ 2 |), 

Xi(P Xl x 2 \ q ) < HPuv) (49) 

K{P Xl x 2 \u q ) < A 2 (iW) (50) 

K(P Xl x 2 \v q ) < MPuv) (51) 

A,(Px 1X2 \ uvq )<HPuv) (52) 

2) The rate pair satisfies 

R 1 >I{U,V;X 1 \X 2 ,Q) (53) 
R 2 >I(U,V;X 2 \X 1 ,Q) (54) 
R 1 +R 2 >I(U,V;X 1 ,X 2 \Q) (55) 

3) There exists (lf(Xi, X 2 ,Q),V(Xi, X 2 .Q)^j such that 
(Ed 1 {U,U),Ed 2 (V,V)) <D. 

or equivalently, 



Kant, s(D) =F(S„a, a ,T>) 
From Theorem [6] we have that 

Sout,2 Q Sout,3 

and therefore 



(56) 



(57) 



ftout,a(D) = P(5 ont , 2 ,D) C 7e out , 3 (D) = P(5 OMt , 3 ,D) 

(58) 

From Theorem we know that 



Sout.2 C S ou tA 



(59) 



Fig. 1. Different sets of probability distributions p(x\, X2\u, v ). 



and 

ftout,2(D) = T{S out . 2 ,T>) C ft out ,i(D) = F{S outA ,T>) 

(60) 

So far, we have not been able to determine whether S out .3 C 
>5oMt,i or iS OM t i C 5 out 3, however, we know that there exists 
some probability distribution p(x\, x 2 \u, v), which belongs to 
Sout.i, but does not belong to 6w.3- For example, assume 
A 2 (A/v) < 1- Let X 1 = {fi(U),S) and X 2 = (f 2 (V),S). 
We note that (X 1 ,X 2 , U, V) satisfy X l -> U V and [/ -> 
V — ► X 2 , i.e., p(xi,X2|u, w) e S out ,i. But, (Jfi,X 2 ) contain 
common information S, which means that X 2 (Px 1 x 2 ) = 1 > 
M{Puv) [7], and therefore, p(xi, x 2 \u, v) S out ^- Based on 
this observation, we can see that introducing S out ,3 helps us 
rule out some unachievable probability distributions that may 
exist in S ou t.\- The relation between different feasible sets of 
probability distributions p(xi, x 2 \u, v) is illustrated in Figure 
1. 

Finally, we note that, we can obtain a tighter outer bound 
in terms of the function D) where the set argument is the 
intersection of <Sw,i and <S „ ti 3, i.e., 

^o«tan3(D) = F{S out .i n 5 outj3 , D) (61) 

It is straightforward to see that this outer bound 7£ ut,in3(D) 
is in general tighter than the outer bound J-(S ou t^i, D) of Tung 
and Housewright. 

IV. Conclusion 



1970s. In this paper, we first proposed an outer bound for the 
rate region, and showed that it is tighter than the outer bound 
of Tung and Housewright. Our outer bound involves some n- 
letter Markov chain constraints, and is not computationally 
practical. To avoid this problem, we utilized a single-letter 
necessary condition for the Markov chain to obtain another 
outer bound for the rate region, which is represented in terms 
of some single-letter mutual information expressions. 
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Tung and Housewright have provided inner and outer 
bounds for the multi-terminal rate-distortion region in late 



